Artificial Intelligence in Ophthalmology: A Comparative Analysis of GPT-3.5, GPT-4, and Human Expertise in Answering StatPearls Questions

Importance Chat Generative Pre-Trained Transformer (ChatGPT) has shown promising performance in various fields, including medicine, business, and law, but its accuracy specialty-specific medical questions, particularly ophthalmology, is still uncertain. Purpose This study evaluates the of two ChatGPT models (GPT-3.5 GPT-4) human professionals answering ophthalmology questions from StatPearls question bank, assessing their outcomes, providing insights into integration artificial intelligence (AI) technology ophthalmology. Methods ChatGPT's was evaluated using 467 bank. These were stratified 11 subcategories, four difficulty levels, three generalized anatomical categories. The answer GPT-3.5, GPT-4, participants assessed. Statistical analysis conducted via Kolmogorov-Smirnov test for normality, one-way variance (ANOVA) statistical significance GPT-3 versus GPT-4 performance, repeated unpaired two-sample t-tests to compare means groups. Results outperformed both GPT-3.5 on except "Lens Cataract" category. differences statistically significant overall, with achieving higher (73.2%) compared (55.5%, p-value < 0.001) humans (58.3%, 0.001). There variations across levels (rated one four), consistently performed better than level-two, -three, -four questions. On level-four difficulty, significantly exceeded that (p = 0.008). Conclusion study's findings demonstrate GPT-4's improvements over Our results highlight potential advanced conversational AI systems be utilized as important tools education practice medicine.